22 research outputs found
Multi-Modal Self-Supervised Learning for Recommendation
The online emergence of multi-modal sharing platforms (eg, TikTok, Youtube)
is powering personalized recommender systems to incorporate various modalities
(eg, visual, textual and acoustic) into the latent user representations. While
existing works on multi-modal recommendation exploit multimedia content
features in enhancing item embeddings, their model representation capability is
limited by heavy label reliance and weak robustness on sparse user behavior
data. Inspired by the recent progress of self-supervised learning in
alleviating label scarcity issue, we explore deriving self-supervision signals
with effectively learning of modality-aware user preference and cross-modal
dependencies. To this end, we propose a new Multi-Modal Self-Supervised
Learning (MMSSL) method which tackles two key challenges. Specifically, to
characterize the inter-dependency between the user-item collaborative view and
item multi-modal semantic view, we design a modality-aware interactive
structure learning paradigm via adversarial perturbations for data
augmentation. In addition, to capture the effects that user's modality-aware
interaction pattern would interweave with each other, a cross-modal contrastive
learning approach is introduced to jointly preserve the inter-modal semantic
commonality and user preference diversity. Experiments on real-world datasets
verify the superiority of our method in offering great potential for multimedia
recommendation over various state-of-the-art baselines. The implementation is
released at: https://github.com/HKUDS/MMSSL.Comment: This paper has been published as a full paper at WWW 202
Let Graph be the Go Board: Gradient-free Node Injection Attack for Graph Neural Networks via Reinforcement Learning
Graph Neural Networks (GNNs) have drawn significant attentions over the years
and been broadly applied to essential applications requiring solid robustness
or vigorous security standards, such as product recommendation and user
behavior modeling. Under these scenarios, exploiting GNN's vulnerabilities and
further downgrading its performance become extremely incentive for adversaries.
Previous attackers mainly focus on structural perturbations or node injections
to the existing graphs, guided by gradients from the surrogate models. Although
they deliver promising results, several limitations still exist. For the
structural perturbation attack, to launch a proposed attack, adversaries need
to manipulate the existing graph topology, which is impractical in most
circumstances. Whereas for the node injection attack, though being more
practical, current approaches require training surrogate models to simulate a
white-box setting, which results in significant performance downgrade when the
surrogate architecture diverges from the actual victim model. To bridge these
gaps, in this paper, we study the problem of black-box node injection attack,
without training a potentially misleading surrogate model. Specifically, we
model the node injection attack as a Markov decision process and propose
Gradient-free Graph Advantage Actor Critic, namely G2A2C, a reinforcement
learning framework in the fashion of advantage actor critic. By directly
querying the victim model, G2A2C learns to inject highly malicious nodes with
extremely limited attacking budgets, while maintaining a similar node feature
distribution. Through our comprehensive experiments over eight acknowledged
benchmark datasets with different characteristics, we demonstrate the superior
performance of our proposed G2A2C over the existing state-of-the-art attackers.
Source code is publicly available at: https://github.com/jumxglhf/G2A2C}.Comment: AAAI 2023. v2: update acknowledgement section. arXiv admin note:
substantial text overlap with arXiv:2202.0938
Boosting Graph Neural Networks via Adaptive Knowledge Distillation
Graph neural networks (GNNs) have shown remarkable performance on diverse
graph mining tasks. Although different GNNs can be unified as the same message
passing framework, they learn complementary knowledge from the same graph.
Knowledge distillation (KD) is developed to combine the diverse knowledge from
multiple models. It transfers knowledge from high-capacity teachers to a
lightweight student. However, to avoid oversmoothing, GNNs are often shallow,
which deviates from the setting of KD. In this context, we revisit KD by
separating its benefits from model compression and emphasizing its power of
transferring knowledge. To this end, we need to tackle two challenges: how to
transfer knowledge from compact teachers to a student with the same capacity;
and, how to exploit student GNN's own strength to learn knowledge. In this
paper, we propose a novel adaptive KD framework, called BGNN, which
sequentially transfers knowledge from multiple GNNs into a student GNN. We also
introduce an adaptive temperature module and a weight boosting module. These
modules guide the student to the appropriate knowledge for effective learning.
Extensive experiments have demonstrated the effectiveness of BGNN. In
particular, we achieve up to 3.05% improvement for node classification and
6.35% improvement for graph classification over vanilla GNNs
Graph Few-shot Learning via Knowledge Transfer
Towards the challenging problem of semi-supervised node classification, there
have been extensive studies. As a frontier, Graph Neural Networks (GNNs) have
aroused great interest recently, which update the representation of each node
by aggregating information of its neighbors. However, most GNNs have shallow
layers with a limited receptive field and may not achieve satisfactory
performance especially when the number of labeled nodes is quite small. To
address this challenge, we innovatively propose a graph few-shot learning (GFL)
algorithm that incorporates prior knowledge learned from auxiliary graphs to
improve classification accuracy on the target graph. Specifically, a
transferable metric space characterized by a node embedding and a
graph-specific prototype embedding function is shared between auxiliary graphs
and the target, facilitating the transfer of structural knowledge. Extensive
experiments and ablation studies on four real-world graph datasets demonstrate
the effectiveness of our proposed model.Comment: Full paper (with Appendix) of AAAI 202
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering
A common thread of open-domain question answering (QA) models employs a
retriever-reader pipeline that first retrieves a handful of relevant passages
from Wikipedia and then peruses the passages to produce an answer. However,
even state-of-the-art readers fail to capture the complex relationships between
entities appearing in questions and retrieved passages, leading to answers that
contradict the facts. In light of this, we propose a novel knowledge Graph
enhanced passage reader, namely Grape, to improve the reader performance for
open-domain QA. Specifically, for each pair of question and retrieved passage,
we first construct a localized bipartite graph, attributed to entity embeddings
extracted from the intermediate layer of the reader model. Then, a graph neural
network learns relational knowledge while fusing graph and contextual
representations into the hidden states of the reader model. Experiments on
three open-domain QA benchmarks show Grape can improve the state-of-the-art
performance by up to 2.2 exact match score with a negligible overhead increase,
with the same retriever and retrieved passages. Our code is publicly available
at https://github.com/jumxglhf/GRAPE.Comment: Findings of EMNLP202
Few-Shot Knowledge Graph Completion
Knowledge graphs (KGs) serve as useful resources for various natural language
processing applications. Previous KG completion approaches require a large
number of training instances (i.e., head-tail entity pairs) for every relation.
The real case is that for most of the relations, very few entity pairs are
available. Existing work of one-shot learning limits method generalizability
for few-shot scenarios and does not fully use the supervisory information;
however, few-shot KG completion has not been well studied yet. In this work, we
propose a novel few-shot relation learning model (FSRL) that aims at
discovering facts of new relations with few-shot references. FSRL can
effectively capture knowledge from heterogeneous graph structure, aggregate
representations of few-shot references, and match similar entity pairs of
reference set for every relation. Extensive experiments on two public datasets
demonstrate that FSRL outperforms the state-of-the-art
Graph-based Molecular Representation Learning
Molecular representation learning (MRL) is a key step to build the connection
between machine learning and chemical science. In particular, it encodes
molecules as numerical vectors preserving the molecular structures and
features, on top of which the downstream tasks (e.g., property prediction) can
be performed. Recently, MRL has achieved considerable progress, especially in
methods based on deep molecular graph learning. In this survey, we
systematically review these graph-based molecular representation techniques,
especially the methods incorporating chemical domain knowledge. Specifically,
we first introduce the features of 2D and 3D molecular graphs. Then we
summarize and categorize MRL methods into three groups based on their input.
Furthermore, we discuss some typical chemical applications supported by MRL. To
facilitate studies in this fast-developing area, we also list the benchmarks
and commonly used datasets in the paper. Finally, we share our thoughts on
future research directions